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DETAILED ACTION 
Specification 

1 . The disclosure is objected to because of the following informalities: 
At 1[[0040], "Fig. 1" should be "Fig. 2". 

At 1I[0046], there needs to be a reference to Step ST706 of Fig. 7. 
At 1|[0064], "an hearing" should be "a hearing". 
At 1I[0066], leg portion "71" should be leg portion "79". (See Fig. 13.) 
At 1|[0066], leg portion "70" should be leg portion "79". (See Fig. 13.) 
At 1|[0067], "may be" should be "may find it". 

At 1|[0073], it is conventional for the Brief Description of the Drawings to be 
placed after the Summary of the Invention. It is suggested that the Brief Description of 
the Drawings be moved to a position between 1|[0026] and 1I[0027], and the U's be 
renumbered accordingly. 

Appropriate correction is required. 

2. The abstract of the disclosure is objected to because it should be one paragraph 
only. Correction is required. See MPEP § 608.01(b). 

Claim Rejections - 35 USC § 103 

3. The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 
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(a) A patent may not be obtained though the invention is not identically disclosed or deschbed as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the phor art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

4. Claims 1 , 3, 4, and 1 8 are rejected under 35 U.S.C. 1 03(a) as being 
unpatentable over Hayashi (JP '250) in view of Sukeda et al. 

Concerning independent claims 1 and 18, Hayashi (JP '250) discloses an 
apparatus and method for making a mobile phone capable of communication between 
an audibly handicapped person and a normal hearing person, comprising: 

"picture capture means for capturing pictures of a first user and a second user" - 
an image is received from video camera input section 9 of a mobile phone; a video 
camera obtains an image for analyzing a motion of a speaker's lips (Abstract; 1I[0007] - 
1I[0008]); implicitly, there are video cameras for both mobile phones when two people 
communicate ("of a first user and a second user"); 

"first conversion means for converting first picture information of the first user, 
whose picture has been captured, into first information that the second user is capable 
of perceiving" - image analysis section 8 of this terminal analyzes an image from video 
camera input section 9, and converts the image into character information and voice 
information corresponding to the image; an image-analysis means analyzes a motion of 
a speaker's lips obtained from the video camera, and changes into text and/or speech 
information (Abstract; 1I[0007] and 1f[0017]); 

"second conversion means for converting second picture information of the 
second user, whose picture has been captured, into second information that the first 
user is capable of perceiving" - image analysis section 8 of this terminal analyzes an 
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image from video camera input section 9, and converts tlie image into cliaracter 
information and voice information corresponding to the image; an image-analysis 
means analyzes a motion of a speaker's lips obtained from the video camera, and 
changes into text and/or speech information (Abstract; 1|[0007] and 1I[0017]); conversion 
is performed for both people communicating with one another; 

"first display means, having a first display screen, for displaying the first 
information, which has been converted, on the first display screen" - a display section 4 
outputs the converted information; text and speech information are displayed on display 
4 as alphabetic character text information (Abstract; 1|[0018]); 

"second display means, for displaying the second information, which has been 
converted, on the second display screen" - a display section 4 outputs the converted 
information; text and speech information are displayed on display 4 as alphabetic 
character text information (Abstract; 1|[0018]); display is performed for both people 
communicating with one another. 

Concerning independent claims 1 and 18, the only element omitted by Hayashi 
(JP '250) is a second display means "having a second display screen capable of being 
placed at an angle different from the angle of the first display screen". Hayashi (JP 
'250) discloses two mobiles phones that translate between speech and image formats 
so that at least one hearing-impaired person can communicate, and although two 
mobile phones can be positioned so that their screens are at different angles to one 
another, implicitly, the angles of the two display screens are not expressly disclosed. 
However, Sukeda et al. teaches an analogous art electronic interpreter for interpreting 
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sentences between a first person and a second person, where tlie electronic interpreter 
can be folded so that displays are arranged in a manner that the display for one user 
and the display for the companion oppose each other. (Column 4, Lines 57 to 62: 
Figure 1) An objective is to provide for interpreting sentences between a first person 
and a second person that is compact for portable use and is simple to use that permits 
a companion of a conversation to reply to the user in a manner understandable to the 
user. (Column 1 , Lines 45 to 56) It would have been obvious to one having ordinary 
skill in the art to utilize the displays of mobile phones capable of making speech 
communication between an audibly handicapped person and a normal hearing listener 
of Hayashi (JP '250) in a configuration where the two displays are capable of being 
placed at an angle different from one another as taught by analogous art of Sukeda et 
al. for a purpose of providing a compact, portable, and easy to use apparatus and 
method of communicating between a user and a companion. 

Concerning claim 3, Hayashi (JP '250) discloses a display section 4 outputs the 
converted information; text and speech information are displayed on display 4 as 
alphabetic character text information ("one of character information . . . .") (Abstract; 
1I[0018]). 

Concerning claim 4, Hayashi (JP '250) discloses that image analysis section 8 
converts the image into voice information corresponding to the image; speech 
information is outputted by loudspeaker 6 as voice (Abstract; 1|[0018]). 
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5. Claims 2 and 5 to 7 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Hayashi (JP '250) in view of Sukeda et al. as applied to claim 1 above, and further 
in view of Basson et al. 

Concerning claim 2, Hayashi (JP '250) discloses converting image information of 
the shape of lips Into text or audio so that a hearing person can communicate with a 
non-hearing person, but omits sound collection means and conversion means for 
converting sound. However, Basson et al. teaches a method and apparatus for 
presenting images representative of an utterance with corresponding decoded speech, 
where a lip reading assistant 100 has a visual feature extractor 102 and an automatic 
speech recognition (ASR) engine 104. ASR engine 104 includes an acoustic feature 
extractor for converting acoustic speech signals into corresponding textual output, and 
is operatively connected to display 108 for visually indicating the decoded textual 
speech to a hearing-impaired user. (Column 3, Lines 19 to 35: Figure 1) An objective 
is to permit a heahng-impaired person to quickly and easily ascertain the relationship 
between body movements used to represent the utterance and corresponding decoded 
speech output. (Column 1 , Lines 45 to 61 ) It would have been obvious to one having 
ordinary skill in the art to provide sound collection means and means for converting 
sound as taught by Basson et al. in an apparatus and method for making speech 
communication between an audibly handicapped person and a normal hearing person 
of Hayashi (JP '250) for a purpose of permitting a hearing-impaired person to quickly 
and easily improve comprehension of an utterance. 
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Concerning claims 5 and 6, Basson et al. teaclies tliat display 108 has first and 
second areas for displaying text 114 and animated images 1 16, 1 18, 120 of facial 
movements. (Column 5, Lines 36 to 41 : Figure 1) Each of the images has time 
information associated therewith which can be subsequently used to synchronize a 
particular image of animation with the decoded speech text from ASR engine 104. 
(Column 4, Lines 54 to 57: Figure 1) 

Concerning claim 7, Basson et al. teaches an embodiment where display 108 
has first and second areas for displaying text 114 and sign information 1 16, 1 18, 120. 
(Column 8, Line 31 to Column 9, Line 5: Figure 4) 

6. Claim 16 is rejected under 35 U.S.C. 103(a) as being unpatentable over Hayashi 
(JP '250) in view of Sukeda et al. as applied to claim 1 above, and further in view of 
Ohgami et al. 

Sukeda et al. discloses an analogous art electronic interpreter, where first and 
second displays 102, 103 have first and second end portions that are connected by a 
hinge so that the first display 102 and the second display 103 may be folded together 
(Figure 1), but omits only "that the second display screen is capable of being twisted". 
However, Ohgami et al. teaches a known configuration for a flat panel display, where 
the display rotates in a twisted position for twisting the flat panel display unit. (Abstract: 
Figures 12 to 15) An objective is to permit an operator to show a display screen to an 
neighboring operator. (Column 1 , Lines 22 to 27) It would have been obvious to one 
having ordinary skill in the art to provide two displays of Sukeda et al. that have an 
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addition capability of twisting relative to one another as taught by Ohgami et al. so that 
one operator can show the display screen to a neighboring operator. 

7. Claim 17 is rejected under 35 U.S.C. 103(a) as being unpatentable over Hayashi 
(JP '250) in view of Sukeda et al. as applied to claim 1 above, and further in view of >Ajbe 
etal. 

Hayashi (JP '250) discloses converting image information of the shape of lips into 
text or audio so that a hearing person can communicate with a non-hearing person, but 
omits storage, reproduction, and trick play means for performing trick play operation on 
stored information. However, Abe et al. teaches a sign language learning system and 
method, where speech, sign-language, and facial expression are input and stored, and 
then output on a display output device and a speech output device. Specifically, a 
display control part of a display output device has input/output areas 714, 716 for 
designating repetition of display and retardation of a display speed. (Column 4, Lines 
50 to 59: Figures 1 , 7, and 8) At least a "slow" speed repetition is "trick play operation". 
An objective is to provide an efficient sign-language learning system and method in a 
form satisfying the requirement of the individual learner. (Column 1, Line 65 to Column 
2, Line 7) It would have been obvious to one having ordinary skill in the art to provide 
storage, reproduction, and trick play means as taught by Abe et al. in an apparatus and 
method for providing speech communication between an audibly handicapped person 
and a normal hearing person of Hayashi (JP '250) for a purpose or providing a feature 
for efficiently learning sign language. 
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8. Claims 8 and 9 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Hayashi (JP '250) in view of Sukeda et al. and Basson et al. as applied to claims 1 and 
2 above, and further in view of Cason. 

Basson et al. teaches utilizing ASR engine 104 to recognize speech, but omits 
the feature of voice activity detection including measurement means for measuring a 
sound pressure level or noise level of sound from a user, setting means for setting a 
threshold value of the sound pressure level or noise level, and execution means for 
causing a conversion means to execute when the measured sound pressure level or 
noise level Is equal to or larger than the threshold value. However, it is well known to 
include a voice activity detector in speech recognition systems so that a speech 
recognition operation is only executed when voice is detected. Specifically, Cason 
teaches a system for detecting voice activity if a measured energy level ("a sound 
pressure level or a noise level") for a frame exceeds a specified threshold. An average 
noise level serves as a floor estimate for a signal. (Abstract; Column 5, Lines 3 to 10; 
Column 6, Lines 1 to 9) Implicitly, a threshold is at least set at some fixed level by 
Cason. An objective Is to detect voice activity for voice recognition systems so that 
periods of silence can be distinguished from periods of speech. (Column 1 , Lines 1 0 to 
18; Column 1 , Lines 48 to 57) It would have been obvious to one having ordinary skill in 
the art to provide a voice activity detection feature as taught by Cason for an automatic 
speech recognition engine of Basson et al. for a purpose of distinguishing periods of 
silence from periods of speech for voice recognition systems. 
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9. Claims 10, 12, and 14 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Hayashi (JP '250) in view of Sukeda et al. and Basson et al. as 
applied to claims 1 , 3, and 4 above, and further in view of Nakamura. 

Basson et al. discloses capturing and associating visual features of lips and text, 
but omits storing and capturing pictures of the lips, and then comparing a picture pattern 
of the lips so as to select one picture pattern corresponding to a captured picture pattern 
of the lips, and extracting character information corresponding to the selected picture 
pattern. That Is, Basson et al. suggests associating images of Hps and text, but does 
not recognize text from lip reading. However, Nakamura teaches a method of 
recognizing speech using a lip image, where parameters are extracted and stored as 
word templates based on outlines of lip motion, a lip pattern is input from a camera, and 
lip data are correlated with word templates so as to recognize speech. (Abstract: Figure 
1) An objective Is to provide speech recognition using a Hp Image that can be 
performed in a small-sized computer having small memory capacity. (Column 1, Lines 
35 to 41 ) It would have been obvious to one having ordinary skill in the art to provide a 
lip reading system of Nakamura In an apparatus and method of Basson et al. for a 
purpose of providing Hp reading that can be performed in a small-sized computer having 
a small memory capacity. 
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Allowable Subject Matter 

10. Claims 11,13, and 15 are objected to as being dependent upon a rejected base 
claim, but would be allowable if rewritten in independent form including all of the 
limitations of the base claim and any intervening claims. 

Conclusion 

1 1 . The prior art made of record and not relied upon is considered pertinent to 
Applicant's disclosure. 

Haskell et a!., Weaver et al., Curry et al., and Liebermann disclose related art. 

Any Inquiry concerning this communication or earlier communications from the 
examiner should be directed to MARTIN LERNER whose telephone number is 
(571)272-7608. The examiner can normally be reached on 8:30 AM to 6:00 PM 
Monday to Thursday. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, David R. Hudspeth can be reached on (571 ) 272-7843. The fax phone 
number for the organization where this application or proceeding is assigned is 571- 
273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
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you have questions on access to tine Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 



/Martin Lerner/ 
Primary Examiner 
Art Unit 2626 
July 10, 2009 



